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PREFACE 


GAO  has  issued  numerous  reports  over  the  years  on 
individual  weapon  systems.  Many  problems  discussed  in 
these  reports  have  a  direct  or  indirect  relationship  to 
the  human  factors  element  and  the  acquisition  process; 
that  is  "what  assurances  are  there  that  weapon  systems 
developed  can  be  operated  and  maintained  by  the  people 
who  must  use  them. " 

The  following  pages  contain  general  guidelines  for 
auditors  to  use  in  evaluating  what  manpower  implications 
were  considered  in  the  design  of  a  weapon  system.  These 
guidelines  do  not  necessarily  include  all  of  the  questions 
that  need  answering.  They  do,  however,  provide  a  frame¬ 
work  from  which  to  begin  and  can  be  modified  or  expanded 
as  experience  in  auditing  a  particular  weapon  system  dic¬ 
tates. 


We  obtained  informal  comments  on  these  guidelines 
from  various  officials  in  the  Department  of  Defense  who, 
because  of  their  knowledge  and  interest  in  the  human  fac¬ 
tors  area,  provided  information  used  in  developing  these 
guidelines.  We  thank  them  for  their  comments  and  believe 
the  guidelines  have  been  greatly  improved  as  a  result  of 
their  input. 

We  encourage  the  use  of  these  guidelines  by  those 
involved  in  the  auditing  profession  and  by  those  having 
day  to  day  responsibility  for  the  administration  and 
management  of  any  particular  weapon  system.  Further 
suggestions  for  revising  and  improving  the  guide  are 
welcome  and  should  oe  addressed  to  me  at  the  Federal 
Personnel  and  Compensation  Division,  U.S.  General 
Accounting  Office,  441  G.  Street,  NW.  ,  Washington,  D.C. 
20548. 
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CHAPTER  1 


INTRODUCTION 


As  highlighted  in  a  recent  GAO  report  ( MASAD-81-26 ,  May  14, 
1981),  GAO's  work  on  weapon  systems  has  culminated  in  24  reports 
on  28  selected  systems.  Those  reports,  issued  from  September 
1980  through  April  1981,  identified  81  issues  in  14  categories 
that  either  have  a  direct  bearing  on  the  weapon  systems'  effec¬ 
tiveness  or  on  the  management  of  the  acquisition  program.  One 
major  issue  concerns  the  manpower,  training,  and  human  factors 
engineering  implications  associated  with  the  design  of  a  weapon 
system:  that  is,  what  assurances  can  be  made  that  (1)  systems 

are  designed  to  meet  the  capabilities  of  people,  (2)  adequate 
numbers  of  people  with  the  right  skills  are  available  to  operate 
and  maintain  the  system,  and  (3)  training  needs  are  identified 
and  met  in  order  to  provide  for  safe  and  effective  operation 
and  maintenance. 

In  another  GAO  report  issued  on  January  29,  1981,  (PSAD-81- 
17),  manpower  problems  were  identified  as  factors  contributing 
to  system  failures.  That  report  said: 

"Limitations  such  as  skill  levels,  proficiency,  availabil¬ 
ity,  environmental  stress,  and  fatigue  of  the  personnel 
who  operate  and  maintain  military  systems  contribute  to 
human-induced  system  failures.  Indications  are  that 
these  types  of  failures  are  quite  high.  New  policy  em¬ 
phasis  on  human  limitations  in  the  design  of  systems  may 
have  a  very  limited  impact  because: 

— Human  factor  specifications,  standards,  and  handbooks 
used  in  designing  and  developing  systems  and  equipment 
do  not  adequately  address  human  limitations. 

— There  are  no  common  collective  methodologies  and  data 
sources  for  use  by  system  designers  in  forecasting  skill 
and  performance  levels  of  future  military  personnel. 

— DOD  (Department  of  Defense)  testing  and  evaluation  poli¬ 
cies  and  procedures  do  not  tend  to  identify  and  resolve 
potei  tial  human-induced  failures  during  the  developmental 
stages  of  the  acquisition  process." 

It  has  generally  been  the  philosophy  that  technology  will 
advance  our  capability  to  meet  the  mission  needs  and  that  man 
can  adapt  to  the  technology.  Designers  of  weapon  systems  often 
do  not  consider  the  capabilities  and  performance  requirements 
of  the  people  who  will  operate  and  maintain  the  systems  in  the 
operational  environment.  It  is  therefore  extremely  important 
that  weapon  system  reviews  include  an  evaluation  of  the  extent 
to  which  DOD  considers  manpower,  personnel,  and  training  needs 
and  capabilities  in  the  weapon  system  acquisition  process. 
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Two  different  approaches  to  reviewing  weapon  systems  can  be 
used.  The  first,  found  in  Chapter  1.  involves  a  review  of  the 
manpower  documentation  required  in  the  decisionmaking  process. 
The  second  approach,  found  in  Chapter  2,  involves  an  analysis 
of  various  organizational  functions  as  they  relate  to  the  weapon 
system  acquisition  process.  Both  contain  general  guidelines  for 
auditors  to  use  in  evaluating  what  manpower,  training,  and  human 
factors  engineering  implications  were  considered  in  the  design 
of  a  weapon  system.  These  guidelines  address  the  question  of 
whether  such  systems  can  be  operated  and  maintained  to  achieve 
readiness  requirements  or  design  capabilities  when  they  become 
available  for  use.  Although  these  guidelines  do  not  necessarily 
represent  all  of  the  questions  that  need  answering,  they  do 
provide  a  framework  which  can  be  modified  as  experience  in 
auditing  a  particular  weapon  system  dictates. 


CHAPTER  2 


REVIEW  OF  REQUIRED  DOCUMENTATION 
GUIDELINES  (APPROACH  #1) 

Final  approval  to  proceed  with  the  development  and/or 
acquisition  of  individual  weapon  system  programs  rests  with  the 
Secretary  of  Defense  or  respective  Service  Secretaries.  In  1969, 
the  Defense  Systems  Acquisition  Review  Council  was  established 
to  review  such  programs  and  provide  the  Secretary  of  Defense 
with  recommendations  concerning  the  status  and  readiness  of 
individual  weapon  systems  to  enter  or  continue  in  the  acquisition 
cycle.  Basically  the  acquisition  cycle  is  broken  down  into  four 
phases  which  are  (1)  Milestone  0 — Program  Initiation,  (2)  Mile¬ 
stone  I — Demonstration  and  Validation,  (3)  Milestone  II — Full 
Scale  Engineering  Development,  and  (4)  Milestone  III — Production 
and  Deployment.  The  Council  plays  an  important  role  in  the 
decisionmaking  process  in  recommending  whether  to  begin  develop¬ 
ment  of  a  new  system  or  proceed  from  one  stage  to  the  next  in 
its  development. 

The  Mission  Element  Need  Statement  (MENS)  is  the  document 
upon  which  the  Milestone  0  decision  is  based.  This  document 
identifies  and  defines  among  other  things  (1)  a  specific  defi¬ 
ciency  or  opportunity  within  a  mission  area,  (2)  the  relative 
priority  of  the  deficiency,  and  (3)  the  general  magnitude  of 
acquisition  resources  that  the  service  is  willing  to  invest 
to  correct  the  deficiency.  An  August  1978  memorandum  from 
the  Assistant  Secretary  of  Defense  (Manpower,  Reserve  Affairs 
and  Logistics)  to  the  Secretaries  of  the  military  departments 
asks  that  manpower  analysis  and  requirements  information  be 
submitted  to  the  Office  of  the  Secretary  of  Defense  (OSD)  to 
support  the  Secretary's  review  of  system  acquisitions.  This 
memorandum  requests  that  the  manpower  information  be  submitted 
to  OSD  in  a  separate  Manpower  Analysis  Paper  (MAP)  or  in  exist¬ 
ing  documents  if  appropriate.  The  data  should  be  submitted  with 
the  MENS  or  each  Decision  Coordinating  Paper  (DCP — document  for 
use  in  Milestone  I,  II,  and  III  decisions). 

The  following  information  presents  guidelines  to  use  during 
each  of  the  program  milestone  review  phases. 

Milestone  0  -  Program 
Initiation  Phase 


(1)  Review  the  MENS  to  determine  if  any  manpower,  training, 
and  human  factors  engineering  constraints  or  opportun¬ 
ities  were  used  to  justify  the  need  for  the  weapon 
system. 
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(2)  Determine  whether  a  MAP  or  other  documents  were  submitted 
with  the  MENS  for  use  in  the  decision  process,  if  not, 
determine  why. 

(3)  If  any  of  the  above  were  submitted,  determine  what  man¬ 
power  data  was  included.  The  documents  should  provide 
an  identification  of  any  broad  manpower  constraints  on 
the  mission  need,  and  a  commitment  to  assess  alterna¬ 
tives  to  reduce  manpower  requirements. 

In  connection  with  this,  determine  (during  this  phase  or  at 
least  in  Milestone  I  phase) 

(a)  whether  data  was  included  that  showed  the  peacetime 
and  wartime  manpower  requirements  compared  against 
a  projected  supply  of  personnel,  throughout  the 
system's  life  span; 

(b)  whether  the  specific  occupational  specialties  and, 
if  possible,  skill  levels  needed  to  operate,  main¬ 
tain,  and  support  the  weapons  system  throughout 
its  life  span  were  identified; 

(c)  whether  estimated  staffing  levels  per  unit  (squad¬ 
ron,  battalion,  ship,  etc. )  to  meet  wartime  and 
peacetime  requirements  were  developed  and  pre¬ 
sented;  and 

(d)  whether  the  lead  times  needed  for  training  or 
obtaining  the  required  number  of  people  were 
presented. 

(4)  During  this  phase,  determine  whether  OSD  or  the  services 
identified  any  manpower,  training,  or  human  factors  engi¬ 
neering  problems  associated  with  previous  similar  systems 
(lessons  learned).  If  so,  determine  what  these  problems 
were  and  if  the  information  was  used  in  the  decision 
process  to  resolve  similar  problems  or  reduce  the  like¬ 
lihood  of  human-induced  system  failures. 

(5)  Deternine  what  procedures  exist  for  insuring  that  needed 
training  programs  are  developed  and  that  training  spaces 
are  reserved  for  the  numbers  of  people  required  to  oper¬ 
ate  and  maintain  the  system.  Determine  at  what  stage  or 
stages  the  service  training  organizations  are  asked  for 
input,  and  how  the  input  affects  the  decision  process. 

If  this  cannot  be  done  in  this  phase,  perform  for 
Milestone  I  phase. 


(6)  Obtain  input  from  training  organizations  and  determine 
if  there  were  assurances  that  needed  training  programs 
could  be  developed.  If  not,  determine  what  tradeoff 
recommendations  were  made.  (Perform  in  Milestone  I 
phase  if  unavailable  in  this  phase. ) 

(7)  Determine  if  there  were  any  lead  times  established  in 
connection  with  the  development  of  training  programs. 
Determine  in  Milestone  I  phase  if  not  done  for  this 
phase . 

(8)  Determine  if  information  regarding  changes  in  estimates 
of  people/skills/aptitudes,  which  may  occur  to  varying 
degrees  during  any  stage  in  the  acquisition  process, 
was  provided  to  the  training  organization  on  a  contin¬ 
uing  basis.  If  so,  assess  any  impact  this  may  have 

on  planned  training.  Determine  in  Milestone  I  phase 
if  not  done  for  this  phase. 

(9)  Determine  if  planning  documents  used  in  the  Planning, 
Programing,  Budgeting  System  address  plans  for  new 
or  improved  weapon  systems  and  the  related  impact  on 
construction  costs  or  possible  training  base  expansion 
requirements. 

(10)  Determine  if  the  contractor  selection  process  (Request 
for  Proposal,  Selection  Panel,  etc.)  included  manpower, 
training,  and  human  factor  engineering  needs.  Determine 
if  the  proposal  evaluation  criteria  included  as  a  speci¬ 
fic  reportable  item  throughout  the  contractual  effort 
information  concerning  manpower,  training,  and  human 
factors  engineering. 

(11)  Determine  if  the  services  provided  baseline  manpower, 
training,  and  human  factors  engineering  data  in  their 
request  for  proposal. 

Milestone  1  -  Demonstration  and  Validation  Phase 

(1)  Determine  whether  a  DCP  was  prepared  and,  if  so,  review 
it  to  determine  whether  it  included  estimates  of  unit 
and  total  program  staffing  levels  for  wartime  and  peace¬ 
time  requirements.  Obtain  and  evaluate  supporting  data 
that  was  or  should  have  been  attached  to  the  DCP. 

(2)  Determine  whether  a  MAP  was  prepared  and  submitted.  If 
so,  determine  what  changes  were  made  from  any  previous 
MAP  and  the  rationale  for  the  changes. 

(3)  Review  the  MAP  and  determine  whether  it  included  infor¬ 
mation  about  optional  manpower  requirements  and  its 
effects  on  operational  capability;  i.e.,  options  in 
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the  number  of  personnel,  skill  (job  proficiency)  levels, 
and  military  occupational  specialties.  Determine  what 
options  were  selected  and  whether  a  rationale  for  selec¬ 
tion  was  presented.  Obtain  and  evaluate  supporting  data 
that  was  or  should  have  been  attached  to  the  MAP. 

(4)  Determine  what  tests  were  planned  for  use  in  assessing 
the  selected  manpower  alternatives  in  the  MAP  or  other 
documents.  Determine  how  the  test  results  were  used 
in  deciding  on  a  particular  option. 

(5)  Verify  whether  the  MAP  or  any  other  document  discussed 
alternative  approaches  to  reducing  manpower  requirements 
or  increasing  productivity.  An  example  of  this  would  be 
a  change  in  maintenance  approach  which  affects  the 
number  of  functions  to  be  performed  and  number  of  per¬ 
sonnel  and  skills  required. 

(6)  Determine  what  test  and  evaluation  procedures  were  or 
are  to  be  performed  to  assess  the  personnel  skill  re¬ 
quirements,  human  factors,  or  training  implications 
of  the  total  system  (e.g.,  performance  under  stress 
or  fatigue,  visual  or  physical  needs,  workload,  apti¬ 
tude,  and  training  required  to  operate  and  maintain). 

Real  world  scenarios  should  be  used  to  identify  people- 
related  problems  with  the  system. 

(7)  Determine  how  the  results  of  demonstration  and  valida¬ 
tion  will  be  used  in  the  decision  process. 

(8)  Review  respective  requests  for  proposals  or  other  simi¬ 
lar  documents  to  determine  if  the  services  included 
statements  addressing  manpower.  Was  service  manpower/ 
personnel/training  information  included  as  part  of 
Government-furnished  information  in  the  contract  package? 

(9)  Determine  if  the  source  selection  process  included 
grading/evaluation  criteria  which  address  manpower 
and  reward  innovative  approaches  aimed  at  reducing 
manpower/training  and/or  personnel  skill  levels. 

Milestone  II  -  Full  Scale  Engineering  Development  Phase 

(1)  During  this  phase,  specific  manpower  estimates  (minimum 
and  maximum  numbers)  should  be  established.  Determine 
whether  the  DCP,  MAP,  or  other  submitted  documentation 
included  this  information.  Obtain  and  review  supporting 
documentation. 

(2)  Assess  the  rationale  for  the  estimates  and  identify 
whether  any  unique  skills  or  specialties  were  estimated 
as  being  needed. 
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(3)  Determine  whether  revised  estimates  of  manpower  require¬ 
ments  were  included  and  the  basis  for  any  revision; 

i.e. ,  demonstrations  from  previous  phase  led  to  alter¬ 
ations  of  previous  estimates. 

(4)  Determine  what  tradeoffs  among  manpower  demand,  person¬ 
nel  supply,  and  training  (quantity,  skills,  specialties) 
were  made.  Obtain  supporting  documentation  and  deter¬ 
mine  rationale  used  in  making  tradeoff  decisions. 

(5)  Determine  whether  requirements  versus  supply  (available 
skills,  particularly  highly  skilled  technicians  in  spe¬ 
cialized  occupations)  were  considered  in  the  decision. 
This  should  include  consideration  of  turnover/attrition 
rates,  specialized  training  required,  and  other  factors 
affecting  the  availability  of  needed  personnel. 

(6)  Determine  whether  man-machine  tradeoff  criteria  were 
used  in  the  development  of  operation  and  maintenance 
work  spaces,  environmental  control  concepts,  and 
control/display/software  design  concepts.  Assess  the 
extent  to  which  developmental  tests  demonstrated  (1)  sys 
tern  conformance  to  human  factors  engineering  design  cri¬ 
teria,  (2)  adequacy  of  training  approach,  (3)  acceptabil 
ity  of  system-imposed  operator  workload,  and  (4)  system 
safety. 

(7)  Determine  if  plans  were  submitted  which  (1)  outline 
procedures  for  acquiring  personnel,  including  Reserves, 
to  meet  wartime  and  peacetime  manpower  requirements, 

(2)  identify  any  staffing  problems  for  any  specialty, 
particularly  those  with  shortages,  and  (3)  evaluate  the 
man-machine  effectiveness  during  acceptance  testing. 

If  such  plans  were  not  submitted  during  this  phase, 
determine  if  they  were  submitted  during  subsequent 
phase  (Production  and  Deployment)  and  if  not,  deter¬ 
mine  why. 

(8)  Determine  if  documentation  identifies  what  tcaining 
programs  and  training  equipment  are  essential  for  each 
skill  progression.  If  documents  do  not  provide  this 
information,  determine  if  the  information  is  included 
in  subsequent  phase  (Production  and  Deployment)  docu¬ 
mentation.  if  not,  determine  why. 

(9)  Determine  what  feedback  process  exists  to  provide 
information  on  manpower/personnel/and  training  short¬ 
comings  found  during  the  various  developmental  and 
operational  testing  that  should  be  considered  in  the 
development  process.  If  process  does  exist,  evaluate 
how  effective  the  process  is. 


7 


Milestone  III  -  Production  and  Deployment  Phase 

(1)  Determine  whether  a  DCP,  MAP,  or  other  document  was 
prepared  for  this  phase  and  review  these  for  manpower 
data.  Obtain  supporting  information. 

(2)  Determine  whether  a  staffing  document  was  prepared 
showing  total  position  requirements  broken  down  by 
specialty  and  skill  level  required.  Determine  whether 
the  document  included  an  analysis  of  such  things  as 
high-demand  workload,  direct  versus  support  work, 
supervisory  assignments,  etc. 

(3)  Obtain  and  review  the  documentation  concerning  infor¬ 
mation  on  specialties  to  determine  whether  shortages 
of  any  required  specialties  have  been  a  problem  in 
the  past,  and  assess  what  plans  there  are  to  overcome 
them. 

(4)  Determine  whether  required  training  equipment  is  avail¬ 
able  so  that  individuals  programed  for  assignment  to 
the  weapon  system  will  have  had  required  training  by 
time  of  deployment. 

Questions  for  the  Contractor/Subcontractor/ 

Program  Manager 

(1)  Determine  whether  the  above  were  asked  to  give  input 
(estimates)  as  to  the  number  of  people/skills/aptitudes 
it  would  take  to  operate  and  maintain  the  system  when  it 
became  deployable.  If  not,  why? 

(2)  If  so,  during  what  stage  (stages)  was  the  input  required? 
Determine  whether  this  information  was  required  within 
established  time  periods  and,  if  so,  was  this  requirement 
met? 

(3)  Determine  who  was  designated  to  receive  this  information 
and  if  it  affected  continuation  to  the  next  phase. 

(4)  Determine  whether  OSD  or  the  services  provided  any  feedback 
to  the  contractor/subcontractor  regarding  submitted  manpower, 
training,  or  human  factors  engineering  data  that  would  affect 
their  estimates  or  design  of  the  system.  Determine  if  the 
feedback  answered  any  questions  raised  by  the  contractor. 

(5)  Determine  whether  the  contractor  was  bound  to  any  manpower, 
training,  or  human  factors  engineering  constraints  in  design¬ 
ing  and  developing  the  weapon  system. 


(6)  Determine  what  dialogue  took  place  between  the  contractor/ 
subcontractor/program  manager/service's  manpower,  training, 
and  human  factors  engineering  organizations  and  OSD  decision¬ 
makers  concerning  these  related  factors. 

(7)  Obtain  contractor/subcontractor  opinions  as  to  effectiveness 
of  the  dialogue. 

(8)  Determine  whether  the  manpower,  personnel,  and  training 
issues  were  considered  for  both  the  system/subsystem  and 
the  total  operational  environment  in  which  it  must  operate, 
including  hostile  and  fatigue-inducing  environments. 

ESTIMATED  STAFF-DAYS  FOR  APPROACH  #1 


Phase 


Staff-days  1/ 


Milestone 

0  -  Program  Initiation 

20 

Milestone 

I  -  Demonstration  and  Validation 

30 

Milestone 

II  -  Full  Scale  Engineering 

Development 

20 

Milestone 

III  -  Production  and  Deployment 

10 

Con tractor /Subcon tractor/Program  Manager 
Total 

1/Assumes  all  work  outlined  will  be  performed. 

are  not  precise  and  will  vary  depending  upon  such  things  as 
the  auditor's  (1)  existing  subject  matter  knowledge,  (2)  cov¬ 
erage  on  each  of  the  individual  guides,  or  (3)  knowledge  of 
organizations  to  visit  to  obtain  needed  information. 
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90 

These  estimates 
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CHAPTER  3 


REVIEW  OF  ORGANIZATIONAL  FUNCTIONS 
GUIDELINES  (APPROACH  #2) 

This  approach  is  an  evaluation  of  the  various  functions 
that,  from  a  manpower,  training,  and  human  factors  engineering 
perspective,  are  important  ingredients  in  the  weapon  system 
acquisition  process.  The  audit  work  outlined  in  this  chapter 
is  not  tailored  to  any  specific  phase  or  milestone  and  does 
not  involve  an  evaluation  of  required  documentation  and  support 
ing  data.  An  auditor  should  bear  in  mind  that  within  a  service 
these  functions  may  cross  organizational  lines.  As  a  result, 
an  auditor  may  be  required  to  visit  several  locations. 

The  functions  presented  for  review  are 

— manpower  requirements  (work  demand  created  by  weapon 
system) , 

— personnel  inventory  (supply  of  personnel  to  do  the  work) 

— training, 

— human  factors  engineering, 

— staffing/assignment, 

— test ing/demonstrat ion/ validation,  and 

— research  considerations. 

Using  this  approach,  an  auditor  should  consider  the  follow 
ing  guidelines. 

MANPOWER  REQUIREMENTS  VERSUS  SUPPLY  OF  PERSONNEL 

A  criticism  of  the  weapon  system  acquisition  process  has 
been  that  needed  manpower  is  neither  available  in  number  or 
required  skills  to  operate  and  maintain  the  system  at  the  time 
the  system  becomes  deployable,  thereby  contributing  to  system 
failures.  The  capability  to  predict  the  availability  of  people 
and  their  aptitudes  and  skills  when  the  system  becomes  deploy¬ 
able  could  help  alleviate  or  lessen  the  magnitude  of  human- 
induced  failures.  To  address  this  issue: 

— Determine  what  manpower  analysis  was  conducted  for  input 
into  the  design/concept/program  initiation  phase. 


10 


-If  manpower  analysis  was  performed,  determine  whether  re¬ 
quirements  (both  wartime  and  peacetime)  were  estimated 
for  the  life  cycle  of  the  system,  and  how  these  estimates 
were  developed. 

-Determine  what  staffing  documents  were  developed  that 
showed  requirements  (wartime  and  peacetime)  by  unit, 
squadron,  ship,  skill,  occupation,  etc. 

-Review  the  staffing  documents  to  determine  if  modifica¬ 
tions  to  the  documents  were  made  and  why.  For  example, 
during  the  demonstration  and  validation  stage,  testing 
results  may  dictate  a  change  in  maintenance  procedures 
to  correct  a  potential  deficiency.  This  in  turn  may 
effect  a  change  in  staffing  requirements.  Such  changes 
can  be  cross-checked  to  the  original  staffing  document. 

-Interview  responsible  officials  to  determine  what 
assurance  they  had  that  the  required  personnel  and 
skills,  as  estimated  in  the  staffing  documents,  would 
be  available  at  time  of  system  deployment. 

-Determine  the  extent  to  which  the  respective  services’ 
organizations  for  determining  manpower  requirements  were 
asked  to  provide  input  to  the  weapon  systems  process 
before  or  during  the  design  (concept/program  initiation 
phase)  and  during  subsequent  phases. 

-Determine  the  extent  to  which  manpower  factors  are  built 
into  the  contractual  agreements  with  the  company  respon¬ 
sible  for  building  the  system  and  the  extent  to  which 
the  company  is  accountable  for  insuring  the  system  is 
built  with  the  user  in  mind. 

-Determine  if  the  program  manager  provides  the  services' 
manpower  organizations  the  estimated  manpower  require¬ 
ments  needed  to  operate  and  maintain  the  system  and  seeks 
feedback  on  whether  the  number  and  skills  will  be  avail¬ 
able  at  time  of  system  deployment.  If  not,  why?  If 
required  numbers  are  available,  determine  what  programs 
are  developed  to  insure  the  retention  of  these  personnel. 

-Determine  what  procedures  exist  to  insure  continuous  dia¬ 
logue  between  program  managers  and  personnel  planning 
organizations  throughout  the  weapon  system's  development. 
Such  procedures  would  be  necessary  to  insure  that 
personnel  planning  organizations  are  aware  of  any  changes 
to  the  weapon  system  which  affect  estimates  on  needed 
manpower  availability. 
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— Determine  if  system  manpower  requirements  are  identified 
specifically  by  system  so  as  to  prevent  loss  of  identity 
in  the  requirements  decisionmaking  process. 

— Determine  how  the  manpower  requirements  were  derived. 

For  example,  were  they  extrapolated  from  a  predecessor 
or  reference  system,  simulation  of  hardware,  or  a  complex 
research  and  development  effort,  etc.  Determine  how  the 
requirements  were  characterized,  i.e.,  by  grade,  years 
of  experience,  aptitude  scores,  etc. 

— Determine  if  alternative  system  concepts  and  hardware 
configurations  were  explored  to  assess  the  impact  on  the 
supply  of  people  by  skill  or  aptitude  level. 

TRAINING 


With  adequate  training  or  retraining,  will  man  be  better 
able  to  adapt  to  the  designed  weapon  system?  Are  the  human 
factor  problems  with  operating  and  maintaining  weapon  systems 
the  result  of  poor  or  inadequate  training  or  poor  quality  of 
people  entering  the  military?  Will  quality  improve?  If  not, 
will  training  or  retraining  provide  for  correcting  the  problems? 
How  can  this  training  or  retraining  be  planned  for  to  insure  that 
people  are  ready  to  operate  and  maintain  the  system  when  deploy¬ 
able?  To  determine  the  extent  to  which  the  services'  training 
organizations  and  procedures  affect  the  weapon  system  design, 
the  following  can  be  done. 

— Determine  and  review  what  procedures  exist  to  insure 
that  the  respective  services  '  training  organizations 
provide  input  on  a  continuous  basis  to  the  weapon  system 
acquisition  process. 

— Determine  and  review  what  procedures  exist  to  insure  that 
training  organizations  are  kept  apprised,  on  a  periodic 
basis,  of  changes  in  concept,  design,  etc.,  which  may 
affect  the  development  of  training  programs  or  training 
position  requirements. 

— Determine  lead  times  needed  for  designing  training  pro¬ 
grams  for  weapon  systems  and  how  classes  will  be  scheduled 
to  insure  all  needed  training  is  received  before  system 
is  deployed. 

— Determine  if  tradeoff  analyses  were  made  concerning 
formal  school  course  content  versus  on-the-job  training 
effectiveness.  Determine  whether  training  device/ 
simulator  requirements  were  explicitly  derived  from 
such  an  analysis.  Determine  if  manpower  requirements 
were  adjusted  to  provide  people  to  conduct  on-the-job 
training. 
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— Determine  how  the  developers  of  the  operations  and 

maintenance  manuals  coordinate  with  the  training  organi¬ 
zations  to  insure  that  manuals  are  not  too  sophisticated 
for  the  estimated  quality  (e.g.,  reading  ability)  of 
people  who  will  be  trained. 

— Determine  length  of  time  needed  for  developing  operations 
and  maintenance  manuals  and  what  input  is  needed  from 
training  organizations  concerning  future  quality  of  people 
who  will  be  using  the  manuals. 

— Determine  what  documentation  on  training  plans  for  oper¬ 
ations  and  maintenance  exists  and  review  the  documentation 
to  insure  it  includes  an  evaluation  of  options  concerning 
equipment  choice  and  design,  technical  publications, 
formal  vs.  on-the-job  training,  unit  training,  individual 
training,  and  simulator  training. 

STAFFING/ASSIGNMENT 

The  development  of  a  weapon  system  could  have  an  impact 
on  staffing,  assignment,  or  classification  procedures  if  part 
of  a  unit  has  to  be  retrained  to  operate  and  maintain  a  new  or 
modified  system.  It  is  therefore  important  that  those  people 
involved  in  making  assignments  are  kept  apprised  of  weapon 
system  development. 

— Determine  what  procedures  exist  to  insure  continuous 
coordination  between  the  weapon  system  acquisition 
process  and  the  personnel  skill  classification  and 
assignment  process. 

— Interview  officials  responsible  for  both  functions  to 
assess  the  impact  weapon  system  development  has  on  staff¬ 
ing/assignment  procedures.  Document  potential  problems. 

— Review  any  documentation  that  shows  how  the  information 
was  requested,  received,  and  fed  back  and  its  timeliness 
in  the  decisionmaking  process. 

TESTING/DEMONSTRATION/VALIDATION 

Before  the  decision  to  enter  full-scale  engineering 
development  is  made,  the  system's  usefulness  should  be  thoroughly 
tested  and  evaluated  so  as  to  get  an  estimate  of  operational  ef¬ 
fectiveness  and  suitability,  including  logistic  supportability . 

To  insure  operational  effectiveness,  human  factors  engineering 
considerations  are  also  important  to  such  estimates.  To  assess 
the  extent  to  which  human  factors  engineering  considerations 
were  included  in  the  testing  and  evaluation  of  the  system,  the 
following  can  be  done: 


— Determine  what  manpower  training  or  human  factors  engi¬ 
neering  constraints  or  problems,  surfaced  from  tests 
performed  on  the  previous  system,  need  consideration 
in  the  new  or  modified  system.  Determine  if  these  con¬ 
straints/problems  were  considered  and  tested  for  and 
what  results  were  obtained. 

— Determine  if  testing  of  the  system  included  any  con¬ 
sideration  of  aptitude  levels,  height,  weight,  stress, 
fatigue,  performance,  etc.,  that  would  affect  system 
effectiveness.  If  so,  determine  how  and  when  this  infor¬ 
mation  was  communicated  to  decisionmakers  and  whether  the 
communication  had  any  impact  on  decisions. 

— Determine  whether  these  test  results  affected  the  esti¬ 
mated  numbers  or  availability  of  personnel  and  skills, 
thereby  effecting  a  needed  change  in  manpower  estimates 
contained  in  staffing  documents.  Determine  if  this 
information  was  communicated,  its  timeliness,  and  results. 

— Determine  whether  testing  results  indicated  the  need  for 
new  or  modified  training  documents,  manuals,  or  programs. 
Assess  results. 

— Determine  whether  testing  identified  the  level  of  person¬ 
nel  proficiency  needed  to  meet  mission  objectives  and 
whether  this  resulted  in  a  plan  for  attaining  and  main¬ 
taining  such  personnel.  Determine  if  this  information 
was  communicated  to  decisionmakers  in  time  to  affect 
decisions. 


RESEARCH/HUMAN  FACTORS 


The  services'  human  factors  engineering  laboratories,  such 
as  the  Army’s  in  Aberdeen,  Maryland,  play  an  important  role  in 
the  weapon  system  acquisition  process  by  researching  and  eval¬ 
uating  manpower  considerations  associated  with  designing,  oper¬ 
ating,  and  maintaining  the  system.  The  role  they  play  is  depend¬ 
ent  upon  the  funding  received  and  number  of  researchers  available. 
Currently,  the  human  engineering  laboratories  get  involved  in 
the  acquisition  cycle  through  industrial  funding  approved  by 
interested  program  managers.  Human  engineering  should  be  a  sig¬ 
nificant  factor  in  weapon  system  design  and  the  auditor  may  wish 
to  consult  the  following  guidelines. 

— Determine  whether  the  funding  methodology  limits  research 
on  weapon  systems  and  if  it  is  more  advantageous  to  con¬ 
duct  research  through  appropriated  funds  versus  indus¬ 
trial  funding. 
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-Interview  responsible  officials  at  the  respective  research 
laboratories  to  determine  the  extent  to  which  they  get 
involved  in  the  acquisition  process. 

-Determine  the  extent  to  which  human  factors  engineering 
regulatory  documents  and  policies  were  employed  during 
the  acquisition  process. 

-Determine  at  what  stage  (program  initiation/  demonstration 
and  validation,  etc. )  researchers  get  involved  and  assess 
whether  they  have  time  to  affect  the  decisionmaking 
process . 

-Determine  the  number  of  researchers  involved  and  their 
average  workload. 

-Sample  researchers  to  determine  their  perspective  on 
the  research  performed,  organizational  difficulties, 
and  suggestions  for  improvement. 

-Determine  the  organizational  alignment  of  the  labs  and 
the  extent  to  which  they  provide  input  to  the  manpower 
and  personnel  organizations  in  the  respective  services.  } 

-Determine  the  attrition/turnover  rate  of  the  researchers  ) 
and  their  reasons  for  leaving  as  well  as  the  average 
amount  of  experience  the  remaining  researchers  have. 

-Determine  why  research  is  conducted  only  through  indus¬ 
trial  funding  and  how  this  affects  the  amount  of  research 
and  type  of  research  conducted. 


ESTIMATED  STAFF-DAYS  FOR  APPROACH  #2 


Function/Organization 

Manpower  Requirements  vs.  Supply 
of  Personnel 

Training 

Staf f ing/Assignment 

Testing/Demonstration/Validation 

Research/Human  Factors  Engineering 
Considerations 

Total 


Staf f-days  1/ 

30 

23 

8 

17 

22 

100 


l/Assumes  all  outlined  work  will  be  performed.  These  estimates 
are  not  precise  and  will  vary  depending  upon  such  things  as 
the  auditor's  (1)  existing  subject  matter  knowledge,  (2)  cov¬ 
erage  on  each  of  the  individual  guides,  or  (3)  knowledge  of 
organizations  to  visit  to  obtain  needed  information. 
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